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SUMMARY 


This  report  contains  preliminary  specifications  for  an  AWG-9  WCS 
maintenance  trainer.  These  were  derived  using  a procedure  described  in 
an  earlier  report  in  this  series.  The  specifications  for  a particular 
application  of  performance-structure-orir.nted  CAl,  such  as  this,  follow 
from  mapping  specific  features  of  task  structures,  contextual  structures, 
and  the  Instructional  st  ategy  to  be  used  into  the  frameworks  of  general 
subsystems  in  CAl  systems  and  of  the  major  interactive  loops  in  these 
systems.  The  resulting  Implications  for  hardware  and  software  specifi- 
cations then  are  developed  by  successive  approximations  until  they  are 
sufficiently  specific  to  guide  hardware  acquisition  and  computer  programming. 
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PREFACE 

This  study  is  another  application  of  Computer  Assisted  Instruction  (CAI) 
to  Navy  training  problems  funded  by  Advance  Research  Projects  Agency  (ARPA), 
performed  by  the  University  of  Southern  California,  and  monitored  by  the 
Naval  Training  Equipment  Center.  The  Radar  Intercept  Officer's  (RIO)  job  as 
noted  in  NAVTRAEQUIPCEN  71-C-0219-1,  was  the  first  technical  area  addressed 
in  the  <nanner  described  herein,  and  the  AW6-9  Weapons  Control  System  Main- 
tenance Technician's  job  is  the  second.  Both  of  these  pace-setting  studies 
were  under  the  very  able  monitorship  of  Dr.  Arthur  Blaiwes  of  the  Naval 
Training  Equipment  Center's  Human  Factors  Laboratory. 

The  preliminary  specifications  described  herein  were  designed  to 
capitalize  on  such  unique  developments  as  the  stand-alone  computer  graphics 
terminal,  the  power  of  minicomputers,  and  the  dedication  to  a training 
philosophy  based  on  the  principle  of  student-initiated  training  scenarios. 
While  these  same  developments  were  used  to  generate  the  RIO  trainer  earlier 
in  this  series,  they  have  only  begun  to  be  used  in  teaching  the  complexities 
of  avionics  maintenance.  In  anticipation  that  principles  learned  in  the 
operational  world  will  transfer  over  to  maintenance  training,  plans  are 
afoot  to  develop  a prototype  trainer  based  on  the  specifications  delineated 
in  this  report. 


WILLIAM  J.  KING,  Ph.D. 
Scientific  Officer 
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SECTION  I 
, I PRODUCTION 

The  objectives  of  this  research  are  to  develop  and  to  field-test  a 
maintenance  trainer,  based  on  TASKTEACH  program  logic  (Rigney,  et  al . , 1972) 
and  a simulated  hands-on  training,  CAI  testbed  (Rigney,  1973).  The  trainer 
will  be  designed  (l)  to  augment  current  system  trainer  facilities  for  train- 
ing built-in-test  (BIT)  operators  with  an  inexpensive,  stand-alone  CAI 
system,  and  {?.)  to  give  technicians  an  understanding  of  the  test  sequences 
in  each  BIT  sequence  in  relation  to  the  functional  and  physical  organization 
of  the  AWG-9  Weapon  Control  System  (WSC)  in  the  F-lU  aircraft. 

As  was  pointed  out  in  an  earlier  report  in  this  series  (Rigney,  et  al., 
197^a),  there  are  many  reasons  why  operational  equipment  is  ill-suited  to 
teaching  Job  skills.  Weapon  systems  are  becoming  prohibitively  expensive 
to  use  for  training.  Operational  equipment  lacks  even  the  most  elementary 
instructional  features.  Maintaining  this  equipment  in  peak  operating  states 
in  schools  is  virtually  impossible.  This  equipment  cannot  intentionally  be 
put  into  malf’inctioning  states  that  would  damage  the  equipment  or  be  hazardous 
to  the  student.  There  is  not  enough  of  it  to  allow  each  student  intensive 
practice  in  performing  Job  skills. 

Under  these  circumstances,  it  is  reasonable  to  seek  alternative  methods 
of  giving  students  troubleshooting  training.  In  the  approach  used  here,  the 
man-machine  interfaces  involved  will  use  interactive  computer  graphics  to 
simulate  "hands-on"  operation.  The  general  techniques  that  will  be  employed 
have  been  described  (Rigney  et  al.,  19T**b).  It  also  would  be  possible  to 
employ  full-scale  front  panel  mock-ups  on-line  with  the  CAI  testbed.  Although 
this  is  not  planned  for  the  first  version  of  the  trainer,  should  the  front 
panel  hardware,  controls,  indicators,  and  panels  become  available  at  a later 
date,  this  alternative  could  be  developed. 

The  principal  advantages  of  this  approach  result  from  substituting  a 
general-purpose  digital  computer,  behind  the  man-machine  interface,  for  the 
complex  electronics  of  the  AWG-9  WCS  and  the  BIT  circuitry.  The  fact  that 
the  actual  circuitry  is  not  behind  this  interface  is,  in  fact,  a distinct 
advantage,  since  all  of  the  disadvantages  of  using  the  actual  equipment  for 
training  are  thereby  avoided.  The  effects  of  any  malfunction,  however 
traumatic  it  might  have  been  to  the  real  equipment,  can  be  described  at  the 
interface.  There  is  no  time  lost  in  changing  "bugged"  weapon  replaceable 
assemblies  (WRA's),  and  no  dependency  on  the  necessarily  very  limited  library 
of  these  to  show  the  student  what  happens  when  the  system  fails.  Beyond  this, 
the  powerful  instructional  features  of  a CAI  system  are  available  for  auto- 
matically administering  and  controlling  training  for  each  student. 
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SECTION  II 


GENERATING  SPECIFICATIONS  FOR  PERFORMANCE-STRUCTURE-ORIENTED  CAI 
A Procedure  for  Generating  CAI  System  Specifications 

Performance-Structure-Oriented  (PSO)  CAI  was  developed  from  con- 
sideration of  the  recent  theoretical  advances  in  cognitive  psychology 
(Rigney,  1971)  and  from  analysis  of  the  structure  of  serial-action  tasks 
(Rigney,  et  al.,  1969),  The  PSO  concept  focuses  on  the  serial-action 
;harac teristics  of  maintenance  tasks.  For  example,  certain  functions  must 
be  performed  before  others,  certain  tasks  must  be  performed  simultaneously, 
or  others  may  be  carried  out  at  any  time.  The  key  to  the  approach  is  that 
such  interdependencies  can  be  cast  into  a general  format  for  programming 
purposes  such  that  powerful  computer  monitoring  and  aiding  of  the  instruc- 
tional task  can  be  realized.  This  conception  of  the  organization  of  perform- 
ance in  terms  of  human  information  processing  functions  was  first  described 
in  Rigney,  et  al.,  (1967).  The  emphasis  in  PSO  CAI  presently  is  on  giving 
students  the  opportunity  to  learn  essential  subskills  and  to  put  them  together 
into  integrated  performance.  It  is  assumed  that  they  will  have  been  exposed 
to  the  necessary  background  knowledge  by  other  delivery  systems. 

Procedures  for  generating  specifications  for  PSO  CAI  were  considered 
in  an  earlier  report  in  this  scries  (Rigney,  1974c),  Some  of  the  general 
concepts  set  forth  there  are  reviewed  here,  as  a matter  of  continuity. 

The  "bases  that  must  be  touched"  are  illustrated  in  figure  1. 

The  analyst  starts  with  available  information  about  what  is  to  be 
done  and  how,  suggested  by  the  arrows  at  the  top  of  the  diagram,  and 
derives  hardware  and  software  specifications  using  procedures  implied 
by  the  interconnected  boxes  in  the  diagram.  When  he  is  finished,  he 
will  have  generated  and  organized  these  two  categories  of  specifications 
for  each  of  the  necessary  elements  of  a CAI  system. 

If  it  is  a tactical  or  operational  job,  the  "subject  matter"  might 
be  air  intercepts.  The  geometry  of  air  intercepts  would  have  to  be 
analyzed,  so  that  the  air  intercept  environment  could  be  simulated.  If 
the  job  is  maintenance  of  devices,  then  the  structure  of  the  device  to 
be  maintained  must  be  analyzed.  Generally,  tactical  jobs  require  quite 
different  data  bases  and  program  logic  than  maintenance  jobs.  Task 
analysis  in  the  diagram  discerns  the  structure  of  serial-action  tasks. 

The  instructional  strategy  is  concerned  with  the  content  and  sequenc- 
ing of  instruction.  The  serial  sequence  of  information  is  the  principal 
means  for  changing  the  behavior  of  the  studeit.  The  instructional  strategy 
will  impact  the  composition  of  the  instructional  sequence:  what  items, 

problems,  drills,  external  feedback,  and  other  instructional  aids  are  used, 
and  in  what  order.  These,  in  turn  will  require  particular  kinds  of  data 
structures  and  storage. 

The  scheduling  of  the  instructional  sequence  will  reouire,  if  done 
adaptively,  a student  monitor  p»"  gram.  This  program,  in  turn,  must  have 
access  to  a student  performance  history  and,  must  contain  or  be  able  to 
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call  upon  some  type  of  optimizer,  and/or  sequence  generator.  (These  CAI 
subsystems  are  illustrated  in  figure  2). 

The  impact  analysis  in  figure  1 abstracts  from  task  and  subject  matter 
analyses  those  features  which  contain  information  for  the  way  CAI  system 
elements  are  designed.  Some  examples  illustrate  this. 

In  the  RIO  trainer,  the  ilr  intercept  task  structure  requires  real- 
time performance  paced  by  a situational  display  showing  fighter  and  bogey 
headings,  speeds,  turns,  etc.  This  requirement  was  met  by  real-time  ani- 
mation using  computer  graphics  programs.  Also  real-time  animation  requires 
a local  processor  and  memory  (in  the  terminal)  because  of  high  refresh  rates 
needed . 

As  another  example,  for  an  AWG-9  maintenance  trainer,  an  important 
feature  of  the  subject  matter  is  the  built-in-test  sequences  that  test 
different  parts  rf  the  system.  Each  of  eight  BIT  sequences  is  composed 
of  a number  of  tests  and  subte  Us , which  are  performed  automatically  and 
in  sequence,  under  program  control  in  contrast  to  simpler  devices  where 
the  technician  selects  the  next  test  to  perform.  The  implication  of  this 
is  that  there  must  be  an  executivi  routine  in  the  trainer  to  sequence  these 
tests,  when  the  student  is  learning  to  use  BIT. 

In  summary,  the  impact  analyses  looks  at  these  tasks  and  contextual 
structures  and  identifies  features  that  would  Impact  on  one  or  more  of  the 
six  necessary  CAI  system  elements.  The  implementation  of  these  elements 
by  hardware  and/or  by  software  then  is  determined  and  specifications  for 
this  Implementation  are  generated  in  one  or  more  of  the  six  boxes  at  the 
bottom  of  the  diagram.  For  example,  program  flowcharts  are  specifications 
for  computer  programs. 

It  is  usually  convenient  to  divide  the  development  of  specifications 
Into  phases.  In  Phase  I,  operations  will  be  mostly  concerned  with  informa- 
tion gathering  and  Integration.  It  may  be  necessary  to  perform  task,  equip- 
ment, and  tactical  analysed  as  part  of  the  development,  or  some  of  thlf  infor- 
mation may  be  available  In  documentation.  In  either  case,  a subject  nw.tter 
expert  is  a practical  necessity  to  select  and  to  interpret  documents  ar d to 
fill  in  gaps  in  information. 

A good  way  to  integrate  Phase  I information  is  to  construct  an  instruc- 
tional flowchart.  This  will  be  an  instructive  exercise,  because  it  forces 
planners  to  put  down  the  surface  structure  of  the  student-program  interaction 
in  detail.  The  instructional  flowchart  will  serve  as  the  principal  guide  for 
Phase  II  operations,  although  it  must  be  supplemented  by  other  sources  of 
information.  Usually,  several  iterations  are  required  before  a flowchart  can 
be  produced  at  a sufficiently  detailed  level  to  be  useful.  Information 
supplementing  the  instructional  flowchart  would  consist  largely  of  the  re- 
sults of  examining  implications  for  each  of  the  general  CAI  subsystems  (see 
figure  2),  in  turn,  of  task  and  contextual  structures  and  of  the  instructional 
decisions’about  how  the  operations  required  for  each  element  will  be  imple- 
mented. During  this  exercise,  the  instructional  flowchart  will  serve  as  a 
reminder  of  features  that  must  be  considered  in  more  detail. 
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In  Phase  II,  the  objective  is  to  describe  the  specific  elements  in 
the  CAI  system  that  will  be  designed  to  do  the  training.  Given  .he  out- 
puts of  Phase  I:  the  instructional  flowchart  and  the  preliminary  delinea- 

tions of  element  operations,  the  analytical  problem  can  be  divided  into 
hardware  and  software  sections,  and  the  effort  can  be  concentrated  on  de- 
veloping specifications  in  the  last  phase  at  the  bottom  of  the  diagram. 

In  summary,  the  whole  process  consists  of  relating  specitic  applica- 
tional  information;  task  and  contextual  structures,  to  general  requirements 
of  CAI  system  elements  and  to  an  instructional  strategy  to  derive  the 
specifications  for  specific  implementations  jf  CAI  system  elements  that 
will  do  the  job  of  training. 

Of  course,  the  analytical  processes  are  not  so  clean  and  symmetrical 
as  figure  1 implies.  For  one  thing,  the  CAI  hardware  system  usually  has 
basic  capabilities  that  would  not  change  unless  some  unusual  variable  in 
the  application  took  on  values  that  exceeded  some  basic  capability;  e.g., 
for  unusual  interface  devices  such  as  front-panel  mockups. 

General  CAI  System  Elements ; CAI  Subsystems 

Any  CAI  system  is  composed  of  several  subsystems,  each  performing  a 
set  of  func-fons  necessary  to  the  total  system  performance.  Functions  of 
these  subsystems  have  been  described  in  earlier  reports  (Rigney,  et  al., 
197iti  197^c ) . Since  the  design  of  the  AWG-9  WCS  maintenance  trainer  is 
based’on  these  general  concepts,  it  is  necessary  to  repeat  a certain  amount 
of  these  earlier  descriptions,  to  achieve  continuity  in  the  sequence  of 
thought  expressed  here.  Tliis  overview  is  referenced  to  figure  2. 


Student— Of  coiu'se,  just  as  is  done  for  other  elements  in  the  CAI  system,  it 
is  necessary  to  develop  "specifications"  for  the  student.  Im-^lic it  assump- 
tions about  student  characteristics  must  be  made  explicit.  There  always  is 
some  target  population  for  whom  the  CAI  system  is  to  be  designed.  Character- 
istics of  this  population  should  be  known  in  relation  to  prerequisites  or 
the  training.  Adapting  to  individual  differences  among  students  is  based  on 
knowledge  about  student  characteristics. 

Rate  of  learning  might  be  predicted  from  a mixture  of  achievement  and 
intelligence  tests.  However,  it  is  preferable  to  use  adaptive  control 
mechanisms  to  sense  important  individual  differences  and  to  adjust  to  them 
as  the  student  is  progressing,  rather  than  to  depend  upon  the  classical 
psychometric  approach,  viz,  batteries  r .'  tests  and  multiple  regression,  to 
try  to  predict  what  "treatment"  should  be  assigned  co  each  student. 

Rt.nfl.^nt-Prograjn  Interface  - This  includes  stimulus^  display , responses,  and 
response  records^  Observe  that  responses  include  trial  ret-ponses  the 
student  may  take  covertly  before  he  commits  himself  to  an  observable 

response. 
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Requirements  here  are  to  identify  the  stimulus  displays,  the  response  I 

input  devices,  and  the  student  data  that  must  be  captured.  For  the  type  of  1 

CAI  system  that  is  under  development  for  the  AW(i-9  WCS,  the  man/machine  | 

interfaces  in  the  Job  environment  are  simulated  with  interactive  computer 
graphics.  Methods  for  doing  this  are  already  well-developed  \t  BTL  (Rigney, 
et  al.  , 197J>b). 

Student  Data — ihis  includes  student  performance  history  and  stadc^c  records. 

The  latter  woiild  include  processed  response  records  plus  other  Information 
about  the  student',  e.g.,  intelligence  test  scores. 

Adaptive  Controller— This  includes  content/mediator  files,  essentially 
storage  of  course  materials  and  auxiliary  instructional  materials,  an 
instructional  sequence  optimizer,  an  instructional  sequence  scheduler,  and 
an  instructional  content  generator.  These  three  instructional  operators 
require  some  explanation.  An  optimizer  would  be  some  method  for  improving 
the  effectiveness  of  the  CAI  system  by  optimizing  learning  ra'\  es  under  some 
set  of  constraints.  Atkinson  and  Paulson  (1972)  have  described  general 
procedures  for  going  about  doing  this. 

The  optimizer  would  identify  an  optimal  instructional  sequence  for 
each  student.  The  composition  and  sequencing  of  instruction  is  the  principal 
r way,  if  not  the  only  way,  to  iniluence  learning  and  retention  in  the  CAI 

system.  The  optimizer  would  control  an  instructional  sequence  scheduler. 

Instruction  must  be  sequenced  by  some  mechanism  and  that  mechanism  must  be 
described  in  the  specifications  for  the  CAI  system. 

The  instructional  content  generator  could  be  either  of  two  types: 
software  that  "makes  up"  the  instruction  on  the  spot  from  simple  descriptive 
data,  as  in  the  TASKTEACH  programs,  or  a team  of  instructors  who  writs 
programmed  instruction  frames,  and  generate  the  entire  sequence  ahead  of 
time. 

It  is  quite  possible,  as  this  labort.tory  has  shown,  for  u computer  to 
ge.ierate  the  instructional  sequence  for  a troubleshooting  problem  step-by- 
step,  while  the  problem  is  being  solved  by  a student,  and  to  do  this  for 
troubleshooting  a wide  variety  of  devices.  It  is  possible  as  this  laboratory 
also  has  demonstrated  (Rigney,  et  al.,  1972),  to  represent  the  structure  of 
tasks  in  simple  data-structures  and  to  provide  a general  program  to  teach 
students  how  to  perform  a variety  of  tasks,  either  allowing  them  to  compose 
their  own  individual  instructional  sequences  or  placing  the  insti’uctional 
sequence  under  program  control. 

For  the  instructional  content  generator,  the  two  major  questions  are: 

"What  is  the  surface  structure  of  the  performance  to  be  taught?"  and  "What 
are  the  content  mediators  that  will  be  required  in  the  training?"  Content 
mediators  are  the  operations  and  concepts  in  the  material  to  be  taught  that 
bridge  between  student  stimulus  and  student  response.  For  example. 
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in  algebra,  the  mathematical  operations  and  mathematical  concepts  required 
to  solve  a quadratic  equation  would  be  content  mediators.  This  tern  is  use 
to  distinguish  between  these  and  learning  and  memory  mediators,  which 
general  operations  the  student  may  perform  on  broad  categories  of  material  to 
improve  rate  of  learning  and  length  of  retention. 

The  content /mediator  files  in  the  adaptive  controller  store  the  material 
that  is  to  be  organized  into  an  instructional  sequence,  or  that  is  alrea^ 
organized  into  some  "lesson"  format.  The  way  this  material 
much  of  it  there  is  to  store  are  important  considerations.  CJJ  requires 
lot  of  storage,  which  usually  takes  the  form  of  disc  files.  The  ran  om- 
access  disc  file  is  a common  and  almost  Indispensable  feature  of  on-line  sys- 
tems. However,  some  CAI  material  may  be  stored  more  economically  in  periph- 
eral devices  in  the  form  of  slides  or  microfiches. 

Instructional  Sequence-As  represented  in  the  diagram,  the  instructional^ con- 
tTnt  would"  hlv7"fc;niready  composed  by  the  generator  and  would  ^ into 

serial  order  by  the  scheduler,  both  in  the  adaptive  controller.  Thus,  the 

Lquenc.  1.  the  output  of  operatore  In  the  adaptive  controller. 

This  sequence  contains  knowledge  of  results,  subject  matter,  and  eternal 
mediators.  It  is  the  "input  tape"  to  the  student.  It  is  necessary  o S" 
tinguish  between  knowledge  of  results  as  something  the  system  Provides  for  the 
stuSnt,  and  internal  feedback  as  something  the  centr^  nervous  system  pro- 
vides^ using  information  that  comes  to  it  both  from  the 
internal  environments.  Knowledge  of  results  provides  some 

internal  feedback,  but  this  information  may  or  may  not  be  usef^.  The  kno  1 
edge  of  res-olts  provided  by  the  system  sometimes  may  be  superfluous. 

Since  this  material  is  at  some  intermediate  difficulty  level  in  a 
roughly  hierarchical  structure,  there  usually  are  a number  of  different  con- 
ce^s,  relations,  and  skills  to  be  learned.  These  must  somehow  be  organized 

into  a sequence  and  presented  to  the  student's  very  ^ 

serial  order.  Thus,  the  fundamental  problem  is  how  to  schedule  the  instruc- 
tional sequence  in  a way  that  will  cover  the  different  concepts,  rules,  and 
operations  to  be  taught,  and  still  lead  to  optimum  learning  and  retention 

rates. 

Manaeement  Informatio_n~Student  data  are  used  to  inform  training 
about  the  progress  of  students  and  classes,  and  to  relate  th^  information 
to  other  variables  that  may  be  of  interest  to  management.  Extensive  stati 
tical  analyses  and  tabulations  may  be  performed,  depending  on  management  s 

requirements . 

Feedback  Loops-The  CAI  system  contains  feedback  channels  that^mustje^used 
to  learn  Ib^  the  student  and  to  learn  about  the  adequacy  of  the  in  t 

tional  sequence. 

It  is  becoming  increasingly  clear,  too,  that  the  human  f , 

also  Contains  many^feedback  loops.  These  evidently 

(Powers,  1973).  The  implications  of  thj..  mode  of  organization  for  the  design 
of  an  instructional  system  are  only  beginning  to  be  realized. 
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Student-CAI  System  Interactive  Loops— CAI  may  be  viewed  as  a series  of  complex 
interactions  between  and  within  the  student  and  the  CAI  system,  occuring  mo- 
ment-by-moment at  the  student -program  interface  (microinteractions),  and 
extending  over  a longer  period  of  time  {macrointeractions ) . This  conception 
is  represented  in  figures  3 and  h. 

Figure  3 is  a highly  generalized  conception  of  these  relationships.  In 
figure  k,  the  two  basic  interactive  loops  have  been  superimposed  on  a simpli- 
fied version  cf  figure  2,  whose  elements  were  described  above.  Again,  these 
figures  have  appeared  in  earlier  reports. 

Microinteractions  are  the  fine-grained,  moment- by-moment  exchanges  of 
information  between  the  student  and  the  CAI  system.  Macrointeractions  are 
involved  with  scheduling  instruction  over  longer  periods  of  time.  These 
interactions  may  be  student-controlled,  program-controlled, or  mixed  initiative. 
Currently,  there  is  much  controversy  over  which  locus  of  control  is  better. 

In  fact,  some  mixture  of  mixed-initiative  control  almost  always  is  employed. 

The  proportions  that  should  be  used  in  the  mixture  depend  on  many  circum- 
stances; maturity  of  the  student,  objectives  of  the  training,  the  incentive 
system,  relative  emphasis  on  instruction  versus  practice,  etc. 

Each  type  of  interaction  becomes  concerned  with  two  kinds  of  structures: 
surface  and  deep.  Surface  structures  are  features  of  the  interaction  that 
are  more  or  less  visible  at  the  student-program  interface.  Deep  structures 
are  processes  which  mediate  the  interactions  and  which  reside  with  the  student 
and  in  the  CAI  system  logic— the  computer  programs. 

Some  of  the  kinds  of  surface  and  deep  structure  features  that  are  consid- 
erations for  planning  micro  and  raacrointeraction  logic  are  suggested  in 
figures  5 and  6.  Obviously,  very  little  is  known  about  deep  structures  in 
the  human  nervous  system  that  mediate  learning  and  memory.  But,  at  least, 
the  interest  of  researchers  has  txirned  to  these  structures  in  recent  years, 
and  some  of  the  resulting  research  does  have  exciting  implications  for 
instructional  technology. 

Deep  stl^^cture  considerations  for  programs  include  questions  of  CAI 
languages  and  their  generality,  of  how  much  interactive  logic  to  put  in  the 
programs  and  how  much  to  distribute  through  the  instructional  sequence,  and 
of  how,  in  general,  to  utilize  the  great  power  and  speed  of  the  digital 
computer  in  education  and  training. 
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This  overview  of  considerations  for  the  design  of  CAl  syste.s  has  been 
reoeated  provide  all  the  Information  relating  to  the  design  of  the 

tur-9  DCS  Ihlnt  nance  Trainer  in  one  place.  Sabsequent  sections,  devot 
to  thif  particuirapplication.  will  take  up  these  general  elements  in  n«re 
detail,  specifying  how  they  will  be  implemented  in  this  trainer.  Fir  . 
Zlever,  questions  of  training  objectives  will  be  considered. 
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SECTION  III 


SPECIFICATIONS  FOR  THE  AWG-9  MAINTENANCE  TRAINER 


Sources  of  Training  Ob.lectives 

Billet  Qualification  Standards — The  Billet  Qualification  Standards  for 
maintenance  technicians  for  the  F-ll,  revised  1 September  1973,  contain 
objectives  for  formal  academic  training  (NAMTRADET),  practical  training 
(Readiness  Squadron)  and  on-the-Job  training  (Operational  Squadron).  A 
minimum  achievement  level  (MAL)  is  defined  for  each  objective.  The  portions 
of  the  Billet  Qualification  Standards  concerning  the  BIT  sequences  for  the 
AWG-9  represent  one  type  of  official  training  objective.  Additional  revisions 
of  these  standards  are  likely  in  the  future  as  the  maintenance  procedures 
eind  this  weapon  control  system  mature. 

Top  Level  Flowcharts  for  Maintenance  Procedures — The  WCS  testing  and  trouble- 
shooting philosophy  is  described  in  flowchart  form  in  figure  7 taken  from 
NAVAIR  01-F1UAAA-2-3-1^ . Again,  this  kind  of  formulation  is  subject  to 
revision  as  more  experience  is  accumulated  with  this  aspect  of  F-lU  mainten- 
ance. Original  assumptions  regarding  the  logistics  of  maintenance,  which 
were  one  basis  for  this  flowchart,  are  likely  to  be  revised.  The  figure 
does,  however,  represent  a procedure!  specification  for  the  use  of  built-in- 
test  sequences. 

Logic  Trees — Decision  trees  for  maintenance  actions  are  being  produced  by 
the  hardware  contractor.  These  are  basically  fault  localization  trees  in 
which  manueil  procedures,  BIT  sequences,  and  BEYOND  BIT  procedures,  e.g., 

SPAM  are  all  specified  as  appropriate  maintenance  actions  at  different 
points  in  the  tree.  These  logic  trees  are  quite  detailed.  At  this  point, 
they  are  in  prepublication  form,  and  are  not  available  for  inclusion  in  this 
rtport.  Basically,  however,  they  are  straightforward  fault-localization 
trees  containing  very  detailed  prescriptions  of  maintenance  actions. 

It  is  worth  noting  that  such  detailed  prescriptions  of  actions  for  the 
technician  to  follow  in  doing  fault  localization  are  the  current  trend  in 
dealing  with  the  human  factor  in  maintainability  of  complicated  weapon 
systems . 

Training  Objectives 

It  is  evident,  from  an  examination  of  information  from  the  above 
sources,  that  the  maintenance  technician  should  be  thoroughly  familiar  with 
the  BIT  sequences,  and  with  the  kinds  of  tests  and  how  they  are  made  in  each 
sequence.  He  alro  should  be  capable  of  operating  each  BIT  sequence;  from 
preinitialization  actions,  initialization  actions,  through  accessing  aegraded 


BIT  HAS  CONTINUED  & 
HAS  DETECTED 
ADDITIONAL 
FAILURES  WITHIN  SE< 


SEQ(s)  = Sequence  (s) 

DP  = Decision  Point 
M = Maintenance 


CHECK  FAILURE 
TABLE  (WIRING) 
TROUBLESHOOT 
SYSTEM  WIRING 


Figure  7.  WCS  Testing  and  Troubleshooting  Philosophy 
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mode  assessment  and  fault  isolation  displays,  to  interpretation  of  dis- 
played information.  Finally,  he  should  be  similarly  familiar  with  digital 
entry  procedures,  e.g.  SPAM,  and  be  capable  of  initiaiting  SPAM  and  using 
the  information  it  displays  in  the  context  of  fault  isolation  trees.  In 
this  regard,  the  objectives  in  the  Billet  Qualification  Standards,  and  in 
the  other  documents  mentioned  above,  will  be  used. 

"He"  in  the  above  paragraph  actually  is  two  technician  levels,  a 
BIT  orerat'r,  and  a more  intensively  trained  senior  technician.  The  latter 
migh.  be  more  likely  to  use  SPAM  or  FLYCATCHER,  as  BEYOND  BIT  procedures 
for  isolating  faults  to  smal..er  fault  areas  than  one  weapon  replacable 
assembly  (WRA) , or  for  isolating  the  faulty  WRA  wher.  BIT  results  require 
supplementary  information. 

Since  PSO  CAI  is  designed  to  give  students  practice  in  performing 
job  tasks,  the  principal  means  for  Identifying  when  .<5tudents  have  achieved 
training  objectives  will  be  time  and  error  measures  made  on  selected  per- 
formance variables,  and  the  principal  means  for  assuring  transfer  to  on- 
tho-job  performance  will  be  simulation  of  the  essential  elements  of  job 
tasks,  AWG-9  man-machine  interfaces,  and  BIT  functions.  In  short,  PSO  CAI 
simulates  the  job  situation  and  assists  the  student  in  learning  to  perform 
job  tasks  until  he  is  able  to  proceed  on  his  own  to  meet  performance  stan- 
dards . 


Subject  Matter  Analysis 

As  implied  in  figure  1,  all  training  problems  have  a subject  matter, 
and  the  latter  in  this  case  is  ^.he  use  and  general  understanding  of  the  BIT 
sequences  in  the  AWG-9.  The  requirement  for  the  analysis  in  this  case  is 
to  identify  those  features  of  the  weapon  system  that  must  be  described, 
without  going  into  additional  detail.  Analytical  techniques  developed  for 
TASKTEACH  will  be  used.  These  result  in  relatively  simple  data  bases  com- 
posed of  lists  of  relevant  features.  As  a part  of  the  analysis,  block 
diagrams  at  a suitable  level  of  detail  for  illustrating  BIT/AWG-9  topography 
will  be  produced.  These  will  be  the  basis  for  interactive  block  diagrams 
implemented  by  the  IMLAC  computer  system.  Some  appreciation  for  the  com- 
plexity of  these  systems  can  be  gained  from  referring  to  the  Appendix. 

Task  Structures 

The  box  labeled  task  analysis  in  figure  1 was  used  in  the  present 
case  of  the  AWG-9  and  BIT,  to  develop  very  thorough  documentation  with  the 
aid  of  the  different  hardware  contractors.  This  includes  detailed  analyses 
of  tasks,  in  the  form  of  lists  of  prescribed  steps  for  maintenance  proce- 
dures. These  documents  have  been  made  available  to  BTL.  Consequently, 
they  are  being  used  as  the  sources  of  information  for  task  analysis. 

The  manual  operating  procedures  for  running  BIT  are  thoroughly  pre- 
scribed. These  procedures  will  be  simulated  by  the  computer  programs  that 
will  track  the  student  while  he  is  operating  on  the  simulated  man-machine 
interface . 
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The  use  of  BIT  is  embedded  in  a broader  context  of  task  prescrip- 
tions, which  do  become  quite  lengthy,  as  in  the  case  of  faul t- local ization 
logic  trees,  and  which  do  require  the  technician  to  refer  to  several 
different,  bulky,  publications  while  doing  line  maintenance.  It  is  an 
objective  of  the  trainer  to  teach  students  to  use  the  standard  Navy  docu- 
mentation that  they  would  have  to  use  in  operating  environments. 

The  Instructiona 1 Strategy 

The  instructional  strategy  used  in  PSO  CAI,  which  is  basically  verti- 
cal iteration  up  and  down  levels  of  complexity,  is  only  one  approach  to 
the  management  of  instruction.  Very  powerful  strategies  for  sequencing 
multiple  strands  have  been  extensively  implemented  by  Chant  and  Atkinson 
(1973),  and  by  Suppes  and  Morningstar  (1970).  The  general  problem  might 
be  characterized  as  how  to  sequence  instructional  information  through  the 
very  limited  human  input  information-processing  channel  to  build  up  effec- 
tive internal  representational  structures,  i.e., mental  models,  in  long- 
term memory  to  support  skilled  performance* 


The  general  features  of  the  PSO  CAI  instructional  strategy  are  illus- 
trated in  figure  8,  which  summarizes  the  major  elements  and  their  relation- 
ships in  this  strategy. 

The  section  of  this  diagram  labeled  front-loading  refers  to  the  points 
for  students  to  enter  the  instructional  system.  It  usually  is  the  case 
that  students  need  preliminary  instruction  before  they  can  start  to  practice 
performing  job  skills.  They  need  to  absorb  information  specifically  relating 
to,  and  supporting  practice  in  performing  job  skills.  One  function  of 
the  front-loading  section  is  to  do  this.  Another  purpose  of  this  section 
is  to  bring  students  "up-to-speed"  with  respect  to  subskills  in  which  they 
are  deficient.  Generally  speaking,  the  three  sections  of  this  diagram  are 
coordinate  with  instructing,  practicing,  and  testing  functions  in  train- 
ing. Of  course,  the  instructional  sequence  may  cycle  through  these  functions 
over  and  over,  and  they  may  be  scheduled  in  the  sequence  in  different  con- 
figurations, which  are  dynamic  features  not  implied  in  the  diagram. 

The  box  1.'  be  led  "instructional  aids"  represents  instructional  tech- 
niques which  would  be  applied  during  practice  to  facilitate  the  development 
of  proficiency.  External  feedback  (knowledge-of-results) , induction  of 
mental  imagery,  and  visual  analogies  of  invisible  processes,  fall  into  th.is 
category. 

According  to  the  conception  of  job  performance  as  the  integrated 
sequencing  of  a collection  of  subskills,  students  will  require  a certain 
amount  of  practice  before  they  can  sustain  this  integrated  performance. 

Until  then,  they  w ’1  have  to  function  at  lower  levels.  The  chain  of 
boxes  labeled  subskill  drills  is  intended  to  indicate  these  levels  of  incom- 
plete integration.  They  would  be  entered  via  remedial  loops.  Some  of  the 
subskills  involved  might  also  be  scheduled  as  front- loading,  particularly 
if  they  are  common  deficiencies  in  the  population  of  students. 
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In  Section  II  of  the  diagram,  coordinate  with  the  testing  function 
instruction,  the  student's  ability  to  sustain  job  task  performance  at 
criterion  levels  is  established,  and  the  student  leaves  the  course.  The 
RIO  trainer  is  an  example  of  the  successful  application  of  this  general 
strategy  (Rigney,  et  al.,  1973b). 

instruct ional  Strategy  for  the  /WG-9  Trainer 

The  instructional  strategy  for  th  AWG-9  Maintenance  Trainer  will 
follow  this  general  plan.  The  two  major  course  elements  that  will  be 
developed  will  be  i.  simulation  of  operational  BIT,  called  Opaque  BIT 
Trainer  (OPBITT)  and  Transparent  BIT  Trainer  (TRANSBITT).  The  ways  in 
which  these  elements  will  be  interrelated  is  shown  in  figure  9. 

In  Stage  I,  the  student  will  be  given  an  overview  of  the  trainer's 
functions.  It  will  be  assumed  that  the  student  will  have  been  exposed  at 
least  to  some  of  the  introductory  audio-visual  materials  used  in  classroom 
training,  and  will  have  some  conception  of  what  BIT  is  all  about.  Then, 
the  student  will  run  OPBITT  to  learn  lo  operate  each  BIT  sequence  in  the 
trainer  and  to  see  the  elements  of  the  final  display  for  each  BIT  sequence. 
The  student  then  will  work  through  all  the  tests  in  the  BIT  sequence,  using 
TRANSBITT,  in  which  the  displays  will  be  interactive  block  diagrams.  Time 
and  error  measures  will  be  used  by  a student-monitor  program  to  indicate 
when  to  transition  the  student  into  Stage  II. 

In  Stage  II,  the  student  will  begin  Lo  practice  interpreting  the 
OPBITT  displays  in  terms  of  the  test  or  tests  that  produced  the  informa- 
tion, and  the  fault  area  or  fault  areas  in  the  AWG-9  from  which  the 
symptoms  emanated.  It  will  be  necessary  for  the  student  to  identify  the 
basic  parts  of  the  BIT  sequence  display.  (This  is  a performance  requirement 
in  the  BQS  manual).  Then,  the  student  will  attempt  to  select  the  particu- 
lar TRANSBITT  block  diagram  at  the  finest-grained  level  of  detail  indicated 
by  these  diagrams.  The  student  should  be  able  to  get  into  the  right  sec- 
tion of  the  top  level  diagram  rather  quickly.  But  he  might  require  more 
practice  before  be  could  access  the  correct  lowest-level  diagram.  Once 
at  the  correct  lowest-level  diagram,  he  could  "rerun"  the  test  to  confirm 
that  the  malfunction  was,  in  fact,  in  a particular  fault  area  and  that  the 
test  output  was,  in  ‘'act,  a particular  decision  point  (DP)  number  or  set 
of  DP  numbers. 

The  student  would  proceed  in  this  way,  through  a set  of  fault  localiza- 
tion problems.  Each  problem  would  be  created  by  simulating  a failure  in 
a particular  fault  area  in  a manner  analogous  to  the  logic  used  in  TASKTEACH 
troubleshooting  programs  (Rigney,  et  al.,  1972).  Since  OPBITT  and  TRANS- 
BITT would  use  the  same  data  base,  the  student  could  transfer  from  one  to 
the  other  inside  a problem.  Time  and  error  scores  would  be  used  to  track 
the  student's  progress  in  learning  to  interpret  OPBITT  displays. 

When  the  student  met  transitioning  criteria,  he  would  move  into  Stage 
III,  where  he  would  be  tested  on  his  proficiency  in  running  BIT  and  in 
interpreting  BIT  displays. 
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The  organization  illustrated  in  figure  9 will  be  used  to  guide  this 
development.  The  single-thread  approach  to  design  of  the  trainer  will 
result  initially  in  data  bases  for  two  of  the  eight  BIT  sequences.  The 
single-thread,  or  vertical-slice,  approach  is  essential  in  early  design 
stages  to  identify  all  of  the  types  of  data  that  will  be  required,  so  that 
program  design  can  start,  without  becoming  swamped  by  an  attempt  to  develop 
an  exhaustive  data  base  from  the  beginning.  An  outline  for  an  instructional 
sequence  covering  more  topics  than  might  be  included  in  the  trainer  is  pre- 
sented, since  some  topics  would  be  taught  by  the  audio-visual  techniques  now 
being  used  by  the  Training  Command. 

In  the  following  outline,  "overview"  is  a combination  of  off-line  tech- 
niques, such  as  sound/slide  shows,  and  on-line  drill.  The  on-line  drill  could 
involve  the  use  of  an  on-line  X-Y  coordinate  tablet  and  an  on-line,  random- 
access  microfiche  viewer.  The  tablet  would  accommodate  schematic  diagrams  and 
other  analogue  representations  in  printed  form,  and  would  accept  pointing 
responses  from  the  student.  Large  amounts  of  technical  information,  in  fact, 
all  the  technical  manuals  used  in  maintenance  of  the  AWG-9,  possibly  excepting 
very  large  diagrams,  could  be  put  on  microfiche,  and  access  could  be  put  under 
computer  control.  Various  types  of  instruction  could  be  provided  in  this  way: 
prescriptions  for  maintenance  actions,  such  as  logic  trees,  or  error-correc- 
ting feedback  , or  background  information.  The  use  of  these  devicjs  will  be 
considered  in  further  planning. 

An  Instructional  Sequence  Outline 

A.  Basic  relationships  and  characteristics  of  BIT. 

1.  Overview  of  AWG-9  and  BIT. 

2.  a.  Theory  of  operation  of  AWG-9. 

b.  Theory  of  operation  of  BIT. 

c.  BIT  displays. 

d.  Implicit  logical  structure  within  BIT  structures. 

e.  Use  of  documentation. 

3.  Operating  Drill  (OPBITT).  The  student  interacts  with  CRT  representa- 
tions of  the  cockpit  controls  and  displays  to  preset  and  to  initiate  BIT 
sequence  1-8. 

U.  Functional  Drill  (TRANSBITT) . The  student  interacts  with  CRT  repre- 
sentations of  functional  block  diagrams  to  learn  the  fault  areas  (e.g.,  WRA's) 
monitored  by  each  BIT  sequence. 

5.  Augmented  Performance.  A combination  o*'  2 and  3 - the  student  runs 
BIT,  receives  indications,  and  attempts  to  localize  the  fault  area.  The 
functional  block  diagrams  serve  as  aids. 


26 


r 


NAVTRAEQUIPCEN  7l<-C-0065-l 

6.  Performance  Testing  (OPBITT).  Similar  to  U except  that  no  augmenta- 
tion is  provided  via  functional  block  diagrams,  and  time  and  error  scores  are 
maintained. 

B.  Specific  BIT  Sequences:  BIT  Sequence  2*. 

1.  Overview  of  BIT  Sequence  2 , 

2.  a.  Theory  of  operation  of  AWG-9  functions  monitored  by  BIT  Seq.  2. 
b.  Theory  of  BIT  sequence  2 (tests,  inputs,  outputs). 

3.  Operating  Drill.  The  student  interacts  with  CRT  representations  of 
the  cockpit  equipment  to  set  up  and  initiate  BIT  Seq.  2. 

U.  Functional  Drill.  The  student  interacts  with  CRT  representations  of 
functional  block  diagrai'<s  to  learn  the  fault  areas  and  functional  blocks 
monitored  by  BIT  Seq.  2. 

5.  Augmented  Performance.  The  student  runs  BIT  Seq.  2,  under  a multi- 
tude of  fault  conditions,  and  attempts  to  localize  the  fault  area  to  desig- 
nated functions/blocks.  He  may  call  for  replacement  of  WRA's  and  checking  of 
bent  connector  pins.  Ff.ults  introduced  by  the  trainer  at  this  stage  do  not 
require  the  use  of  BIT  sequence  digital  entries. 

6.  Performance  Testing.  Same  as  A5. 

C - H.  Specific  BIT  Sequences:  BIT  Sequences  3-8;  same  instructional 

sequence  as  B. 

I.  Use  of  SPAM  (scope  to  be  defined  later). 

1.  Overview  of  use  of  SPAM. 

2.  Operating  Drill.  The  student  interacts  with  CRT  representations  of 
the  cockpit  equipment  to  run  the  eight  BIT  sequences,  followed  by  the 
appropriate  SPAM  procedures. 

3.  Functional  Drill.  The  student  interacts  with  CFiT  representations  of 
functional  block  diagrams  to  learn  the  functions  monitored  by  SPAM. 

U.  Augmented  Performance.  The  student  runs  BIT,  receives  indications, 
and  attempts  to  identify  the  faulty  WRA  and  block.  Problems  simulated  by  the 
trainer  in  this  stage  require  use  of  SPAM. 


•It  is  recognized  that  BIT  sequence  1 is  atypical  of  subsequent  BIT  sequences, 
and  that  it  would  require  special  treatment  if  implemented  in  the  trainer. 

For  these  reasons,  implementation  of  this  sequence  in  the  trainer  will  be 
deferred.  This  outline  applies  more  closely  to  BIT  sequences  2-8. 
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5 Performance  Testing  (OPBITT).  Similar  to  4 except  that  no  augmen- 
tation is  provided  via  functional  block  diagrams,  and  time  and 
error  scores  are  maintained. 

B.  Specific  BIT  Sequences;  BIT  Sequence  2*  . 

1.  Overview  of  BIT  Sequence  2 . 

2.  a.  Theory  of  operation  of  AWG-9  functions  monitored  by  B^T  sequence  2. 
b.  Theory  of  BIT  sequence  2 (tests,  inputs,  outputs)* 

3.  Operating  Drill  - The  student  interacts  with  CRT  representations  of 
the  cockpit  equipment  to  set-up  and  initiate  BIT  sequence  2. 

h.  Functional  Drill  - The  student  interacts  with  CRT  representations 

of  functional  block  diagrams  to  learn  the  fault  areas  and  functional 
blocks  monitored  by  BIT  sequence  2. 

5.  Augmented  Performance  - The  student  runs  BIT  sequence  2,  under  a 
multitude  of  fault  conditions,  and  attempts  to  localize  the  fault 
area  to  designated  functions/blocks.  He  may  call  for  replacement 
of  WRA's  and  checking  of  bent  connector  pins.  Faults  introduced 

by  the  trainer  at  this  stage  do  not  require  the  use  of  BIT  sequence 
digital  entries. 

6.  Performance  Testing  - same  as  A5. 

C - H.  Specific  BIT  Sequences;  BIT  Sequences  3-8;  same  instructional 
sequence  as  B. 

I.  Use  of  SPAM  (scope  to  be  defined  later). 


1.  Over-.'iew  of  use  of  SPAM. 

2.  Operating  Drill  - The  student  interacts  with  CRT  representations  of 
the  cockpit  equipment  to  run  the  eight  BIT  sequences,  followed  by 
the  appropriate  SPAM  procedures. 

3 Functional  Drill  - The  student  interacts  with  CRT  representations 
of  functional  block  diagrams  to  learn  the  functions  monitored  by 

SPAM. 


Augmented  Performance  - The  student  runs  BIT,  receives  indications, 
and  attempts  to  identify  the  faulty  WRA  and  block.  Problems  simu- 
lated by  the  trainer  in  this  stage  require  use  of  SPAM. 


*It  is  recognized  that  BIT  sequence  1 is  atypical  of  subsequent  BIT  sequences, 
and  that  it  would  require  special  treatment  if  implemented  in  the  framer. 

Tor  Jhese  reasons,  implementation  of  this  sequence  in  the  trainer  will  be 
deferred.  This  outline  applies  more  closely  to  BIT  sequences  2 8. 
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5.  Performance  Testing.  Same  as  U but  without  error-correcting  feed- 
back. 

J.  Strategy  of  Applying  the  Complete  BIT  System  to  Maintenance  of  the  AWG-9. 

1.  Overview  of  AWG-9  WCS  testing  and  troubleshooting  philosophy. 

2.  Operating  Drill.  The  student  interacts  with  a CRT  representation 
of  "WCS  testing  and  troubleshooting  philosophy"  flowchart  to  learn  the  proper 
application  of  the  BIT  procedure. 

3.  Functional  Drill.  None  at  this  level.  If  needed,  return  to  A-J 
drills. 

U.  Augmented  Performance.  The  student  runs  BIT,  receives  indications, 
and  atten?>ts  to  identify  the  f&vlt  area.  He  may  call  for  replacement  of 
WRA's,  checking  for  bent  pins,  rerunning  BIT  sequences,  and  running  SPAM, 
jjg  vixi  receive  error-correcting  feedback,  based  on  standard  documentation. 

5.  Performance  Testing.  Same  as  U,  but  without  feedback. 


Block  Diagrams  of  Instructional  Sequence. 

The  instructional  sequence  is  diagrammed  below: 


Figure  10.  Subject-matter  Modules  for  Macrointeractions 


Each  of  the  instructional  modules  A through  J would  contain  a common 
sequence,  as  follows: 


- 

■ ~ — — - •• 

' *■' 

OVER- 

VIEW 

OPER- 

ATING 

FUNC- 

TIONAL 

AUG- 

MENTED 
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PERFORM- 

ANCE 

— > 
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i 

DRILL 
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DRILL 

J 

PERFORM- 

ANCE 

TEST 

Figure  11.  Sequence  of  Drills  Within  a Subject-matter  Module 
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Software  Implications  Analysis 

The  generative  approach  to  CAI  (Carbonell,  1972i  Rigney,  et  al.,  1970) 
will  he  used.  In  this  approach,  the  computer  progreims  generate  the  inter- 
action with  the  student  from  relatively  simple  data  bases.  In  general  terms, 
for  this  type  of  trainer,  which  requires  simulation  of  job  tasks  and  man/ 
machine  interfaces  in  the  Job  environment,  programs  will  be  required  for  the 
following  fvmctions: 

Man/Machine  Interface  Simulation — This  will  be  done  with  techniques  that  have 
been  developed  at  BTL  (Rigney,  et  al.,  197Ub)  for  creating  interactive  com- 
puter graphics  for  simulating  front  panels  and  for  displaying  interactive 
block  diagrams.  Certain  aspects  of  this  type  of  program  development  have 
been  greatly  simplified  by  an  interpreter  which  generates  code  for  graphic 
displays  while  the  operator  is  drawing  these  graphics;  using  function  keys 
and  the  cursor  on  the  graphics  terminal. 

Student-Program  Interactions— While  the  analogue  representational  aspects  of 
these  will  be  supported  by  the  interactive  computer  graphics,  the  digital 
representational  aspects  will  require  a series  of  routines  to  implement  the 
sequence  of  drills  described  above,  between  and  inside  each  block  of  subject- 
matter  (figures  10  and  11).  These  programs  will  contain  logic  very  similar 
to  that  in  TASKTEACH  troubleshooting  programs  for  simpler  equipment.  However, 
the  much  greater  complexity  of  the  AWG-9/BIT  systems , the  fact  that  a BIT  sys- 
tem is  included  and  that  the  maintenance  concept  prescribes  detailed  mainten- 
ance procedures  for  the  technician  to  follow,  all  dictate  comprehensive  addi- 
tions to  that  logic.  Consideration  of  requirements  for  sustaining  the  micro- 
interactions described  above  has  led  to  the  following  general  descriptions 
of  program  module  functions.  The  discussion  includes  suggestions  for 
sequencing  criteria  and  for  student  data  that  might  be  used  for  performance 
histories  for  an  adaptive  controller  (see  figure  l). 

Program  Module  Functions— The  five  basic  program  modules  will  now  be  charac- 
terized in  more  detail. 

a Overview.  This  would  consist  of  several  routines,  all  operating  on 
the  same  data  base.  In  the  most  general  of  characterizations,  the  data 
needed  can  be  viewed  as : 

(l).  A list  of  elements,  each  element  possessing  a list  of  properties. 


Element  name 

BIT  monitoring  sequence 

Decision  point 

WRA 

Name  of  sub-test  in  BIT  sequence 
Page  number  in  documentation 


For  example: 


Element  I 


Subband  I 

5 

110 
039 

Doppler  Processor 

V.  117.5 
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Element  II^ 


Subband  II 


(2).  A group  of  interactive  diagrams  relating  the  elements  in  a 
meaningful  way.  (Preliminary  versions  of  these  diagrams  are  being  prepared 
for  BIT  sequences  3 and  5). 


The  program  then  could  execute  numerous  mapping  and  classification  drills, 
as  follows: 


1.  AWG-9  Organization.  The  trainer  displays  the  element  diagrams 

without  WRA  or  other  fault  area  names.  In  drill  a the  student  points  to  the 

various  blocks  euid  the  trainer  displays  the  names.  In  drill  b the  trainer 

presents  the  names  in  random  order  and  the  student  points  out  the  blocks  in 
the  diagram.  This  is  really  a drill  related  to  orgsinization  rather  than 
names,  since  the  student  must  associate  a given  name  with  an  unlabeled  block 
in  a diagram. 

2.  BIT  Sequence  Organization.  The  trainer  displays  the  diagrams,  with 
WRA  or  other  fault  area  names.  In  drill  a the  student  points  to  the  various 
names  and  the  trainer  presents  the  BIT  sequences , and  the  student  points  out 
the  appropriate  blocks.  This  would  include  sequential  dependencies  across 
BIT  sequences. 

3.  Functional  Organization.  This  is  similar  to  drill  1,  except  that 
the  student  is  drilled  in  associating  functional  block  names  to  unlabeled 
blocks  within  a WRA. 

U.  Test  Theory.  The  trainer  displays  the  diagrams  with  WRA  or  fault 
Eu:ea  names  and  functional  block  names.  In  drill  a the  student  points  to  the 
names  of  a particular  test  (within  a BIT  sequence),  and  the  trainer  high- 
lights the  blocks  monitored,  the  input  point,  and  the  point (s)  of  measurement. 
In  drill  b,  the  trainer  presents  the  test  names  and  the  student  indicates  the 
blocks  involved. 

The  program  described  above  can  exhibit  great  generality,  since  the 
property  lists  associated  with  elements  can  be  quite  flexible.  Further,  the 
general  interaction  is  consistent;  the  program  first  performs  the  mapping  of 
a particular  characteristic  onto  the  diagrams,  then  the  roles  are  reversed 
and  the  trainer  serves  as  a checker,  supplying  answers  when  the  student  is 
stumped. 


Finally  a simple  set  of  parameters  can  be  provided  with  the  data  base 
which  governs  the  progression  of  the  student  through  the  various  drills. 

Each  mapping  task  (drill  b)  would  have  an  associated  time  and  error  criterion 
which  must  be  achieved  before  progressing  to  the  next  drill.  Of  course,  the 
student  would  have  greater  control  over  rate  and  repetition  of  the  a portions 
of  each  drill. 
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b.  Operating  Drill.  The  data  base  for  the  operating  drill  would  con- 
sist of  front-panel  conditions,  symptoms  at  indicators,  various  display 
lists  (D-lists)  which  would  produce  "pictures"  on  the  CRT,  and,  possibly, 
small  task  structures.  Any  operating  drill  can  therefore  be  implemented 

by  accessing  a small  list  of  pointers  which  specify  where  these  data  are 
stored  on  a storage  device  (disk).  Upon  loading  these  data  (.02  - . !j 
seconds)^  the  operating  drill  program  would  enter  a "ready"  loop,  awaiting 
input  from  the  student.  Student  inputs  would  consist  primarily  of  control 
manipulations  via  a light  pen.  The  trainer  would  simulate  the  operational 
equipment  and  record  pertinent  data  regarding  student  performance.  The 
ooerational  drill  can  be  considered  a portion  of  Opaque  BITT,  which  treats 
surface  structures  in  the  man/machine  interface  (figure  3).  It  would  in- 
clude digital  entry  operations. 

c.  Functional  Drill . During  the  functional  drill,  the  student  inter- 
acts with  functional  block  diagrams  displayed  on  the  CRT.  The  student 
practices  identifying  input/output  points  and  functions  tested  for  the  sub- 
tests within  a BIT  sequence.  Th-e  diagrams  are  dynamic  in  the  sense  that 
student  inputs  and  program  responses  will  affect  them.  These  diagrams  will 
serve  to  integrate  information  pertaining  to  BIT  sequence  test  variables 
with  fault  areas  and  functional  blocks  in  the  AWG-9. 

d.  Augmented  Performance.  During  augmented  performance, the  trainer 
simulates  the  observable  aspects  of  the  AWC-9  hardware  and  the  BIT  soft- 
ware. In  addition,  it  injects  malfunctions  into  the  simulated  environ- 
ment. The  student  attempts  to  perform  the  maintenance  task  using  the  nor- 
mally available  documentation,  but  he  may  rely  on  the  functional  block 
diagrams  provided  in  the  functional  drill.  The  student's  objective  is 

to  become  independent  of  these  augmenting  features,  and  perform  exactly 
as  he  will  when  in  the  actual  maintenance  environment. 

Research  by  others,  and  direct  experience  by  this  laboratory, 
indicates  rather  clearly  that  optional  features  under  student-control 
are  not  effectively  used  by  students.  This  may  be  because  the  material 
was  not  truly  helpful.  Or,  perhaps  the  student  is  not  in  a position  to 
decide  if  some  material  which  he  has  not  seen  would  be  useful  to  him.  In 
any  case,  the  augmenting  material  here,  the  functional  block  diagrams, 
do  not  represent  new  information  which  was  previously  missing.  Rather, 
the  diagrams  should  serve  to  remind  the  student  of  relationships  already 
covered  during  the  functional  drills,  or  perhaps  to  illustrate  consequences 
of  certain  relationships  in  light  of  a particular  malfunction  situation. 
Thus, it  appears  that  the  diagrams  might  best  be  integrated  into  student 
performance  by  requiring  the  student  to  "explain"  each  problem  in  relation 
to  the  diagram.  This  might  occur  at  the  conclusion  of  each  problem,  or, 
if  possible,  would  be  done  within  a problem.  The  latter  approach  has  the 
advantage  of  closer  tracking  of  the  student,  allowing  program  intervention 
if  the  student  embarks  on  an  inappropriate  activity. 

In  addition  to  using  the  diagrams  to  increase  the  fidelity  of  the 
trainer's  evaluation  of  student  performance,  they  also  would  be  available 
for  program  or  student  control  during  each  problem.  Quite  possibly  the 
student  also  could  return  to  certain  portions  of  the  functional  drill  if 

3? 


r 


NAVTRAEQUIPCEN  7>4-C-0065-i 


! 


i 


he,  or  the  trainer,  sensed  a deficiency  in  some  area.  These  options  will 
be  dealt  with  in  the  design  of  the  macrointeraction  loop  (see  figure  I4). 

e.  Transitioning  to  Performance  Testing.  The  only  real  difference  ^ 

between  augmented  performance  and  performance  testing,  in  this  planned  j 

trainer,  is  that  the  interactive  block  diagrams  would  not  be  provided.  i 

Thus, at  some  point  the  student  must  explicitly  terminate  the  augmented 
performance  stage  and  enter  performance  testing. 

It  is  proposed  that  the  trainer  will  determine  when  some  prespecified 
criteria  have  been  satisfied  by  the  student  during  augmented  performance,  I 

and  will  indicate  that  he  may  proceed  to  the  performance  testing  stage.  3 

If  desired,  under  certain  conditions  the  student  might  continue  in  the 
augmented  performance  stage.  He  might  continue  until  all  problems  have 
been  worked  or  his  allotted  computer  time  has  been  expended. 


The  criteria  for  transitioning  to  performance  testing  v,  . be  variables 
which  can  be  set  by  the  Training  Command.  The  trainer  will  aiaintain  a number 
of  performance  measures  from  which  the  using  facility  can  choose.  Typical  of 
these  would  be: 


1.  Number  of  problems  attempted 

2.  Number  of  problems  solved 

3.  Problems  solved  7 problems  attempted 
I*.  Average  solution  time  per  problem 

5.  Average  number  of  VfRA  replacements  per  problem 

6.  Expected  replacement  time  per  problem  (this  is  the  time  reejuired 
to  make  the  replacements  requested  by  the  student,  as  taken  from 
flight  line  history) 

7.  Total  expected  repair  time  (#•<  + #6  for  each  problem). 

The  transition  rule  might  then  be  formulated  as; 

number  of  problems  attempted  - 30 

problems  solved  -f  problems  attempted  ^ .85 
average  solution  time  per  problem  ^20  min. 

expected  replacement  time  per  problem  ^1.7  hours. 

Again,  these  are  considerations  for  the  design  of  the  macrointeraction 
loop.  When  this  loop  is  under  program  control,  the  adaptive  controller, 
consisting  of  several  programs,  would  do  this  controlling. 

Adaptive  Controller — Typical  variables  which  might  be  included ^in  student  data 
were  listed  above.  The  point  was  made  that  criterion  values  of  these  should 
be  adjustable  by  the  Training  Command.  If  this  is  desired,  these  variables 
also  would  be  used  for  trials-to-criterion  logic  automatically  implemented 
by  a computer  program.  This  approach  was  taken  in  the  RIO  trainer  iRigney 
et  al.,  1973b).  This  implementation  requires  several  additional  routines, 
for  student  data  capture,  data-analysis , and  for  monitoring  student  progress 
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(see  figure  2).  Together,  these  routines  would  optimize  instruction  in 
terms  of  some  goal,  e.g.,  minimize  time  to  complete  the  course,  given 
fixed  performance  criteria,  for  a population  of  students.  An  important 
consequence  of  trials-to-criter ion  logic  is  that  it  results  in  reduced 
variance  in  student  proficiency  at  the  output  end  of  the  course. 

Hardware  Impl icat ions  Analysis 

The  outstanding  computer  liardware  requirement  for  CAI  is  storage. 

The  IMLAC  PL'S-4  intelligent  terminal  will  have  16K  words  of  relatively 
fast-access  core  memory  (.9  microseconds  minimum)  for  running  data- 
processing  and  display-processing  programs.  This  will  be  augmented  by 
floppy-disk  storage  of  256K  words.  Since  these  disks  can  be  very  quickly 
and  simply  inserted  or  removed  from  the  disk  controller,  they  also  afford 
unlimited  off-line  storage. 

Processor  speeds  are  not  considered  to  be  a constraint  in  this  self- 
standing system.  Both  the  display  processor  and  the  central  processing 
unit  (CPU)  are  dedicated  to  one  student  at  a time.  With  a display  refresh- 
rate  of  40  Hz,  and  a 900  nanosecond  cycle  time,  there  are  very  few  occasions 
when  display  flicker  would  occur.  An  earlier  version  of  this  terminal  with 
a 1.8  microsecond  cycle  time  has  been  used  extensively  for  a real-time 
performance  training  problem  without  experiencing  any  flicker. 

The  basic  system;  intelligent  terminal  with  light  pen,  floppy-disk 
controller,  and  general  purpose  10  interface,  will  constitute  the  initial 
system.  Several  different  peripheral  devices  are  available  as  add-ons, 
controlled  through  the  10  interface:  a random-access  slide  projector, 

digital  voice-synthesizer,  microfiche  projector,  and  a tablet.  The  micro- 
fiche projector  would  be  attractive  for  storing  the  voluminous  technical 
ma.Tuals  associated  with  the  AWCf-9/BIT  system,  and  for  making  the  information 
readily  accessible  under  computer  control.  Some  consideration  will  be 
given  to  this  possibility. 

Scope  o f Deve lopment 

The  trainer  to  bo  developed  will  contain  the  program  modules  des- 
cribed earlier.  These  modules  will  be  sufficiently  general  that  they 
can  function  for  any  of  the  BIT  sequences  two  through  eight. 

In  addition  they  will  function  for  the  overall  BIT  stage  (a)>  the 
testing  and  troubleshooting  philosophy  stage  (j),  and  the  memory  inspec- 
tion mode  of  SPAM  ( i) . Initially,  a data  base  for  BIT  sequence  S and  a data 
base  for  BIT  sequence  3 will  be  prepared.  The  data  for  sequence  3 will  be 
that  required  to  transition  to  sequence  5. 

The  trainer,  as  ir.itially  put  into  the  field,  will  include 
running  two  of  the  eight  BIT  sequences,  performing  digital  entries  for 
one  BIT  sequence,  and  the  entire  instructional  strategy  which  applies  to 
all  BIT  sequences. 
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At  this  point,  consumer  acceptance  of  the  trainer  will  be  tested, 
and  consumer  suggestions  for  modifications  will  be  taken  into  considera- 
tion. Expansion  of  the  trainer  to  teach  BIT  sequences  2,  6,  7,  and  8 
could  be  done  by  adding  data  modules  for  those  sequences. 

Since  the  preliminary  specifications  that  have  been  described  in 
this  report  are  derived  from  a concept  of  trainers  and  of  training  that 
differs  from  the  "big  simulator"  approach  now  commonly  encountered,  plans 
for  testing  of  this  particular  trainer  include,  ultimately,  cost-effectiveness 
comparisons  with  appropriate,  currently  existing  examples  of  this  approach. 
Performance  criteria  such  as  those  listed  on  preceding  pages  will  be  used 
for  evaluating  training  effectiveness.  Estimating  the  cost  of  bringing 
students  up  to  criterion  levels  of  performance  will  require  formulation  of 
a multivariate  cost-effectiveness  model. 
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APPENDIX  A 


This  is  a brief  overview  of  the  BIT  system  in  the  AWG-9  WCS  in  the 
F-14.  This  material  was  abstracted  from  several  Navy  publications. 

All  diagrams  are  from  these  publications.  It  is  presented  here  merely 
for  illustrative  purposes.  ^ 


The  operation  of  BIT  by  the  maintenance  technician  is  performed 
from  the  rear  cockpit  of  the  aircraft,  using  tactical  displays  and  controls 
which  also  serve  for  controlling  BIT.  The  principal  displays  used  for 
this  purpose  are  the  Tactical  Information  Display  (TID),  and  the  Detail 
Data  Display  (DDD) . In  addition,  many  controls  have  integral  lights 
which  turn  on  when  the  control  is  activated.  The  principal  controls  the 
maintenance  technician  used  are  grouped  on  panels:  the  Computer  Address 

Panel,  the  TID  panel,  the  Sensor  Control  Panel,  the  DDD  panel,  and  the 
Hand  Control  Panel. 
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Figure  A-12.  WCS  Equipment  Location 
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Rack  numbers,  WRA  numbers,  abbreviations,  and  names  o£  the  units  shown 
in  figure  12: 

RIGHT  SIDE 

963  RACK: 


730  - MAS  - Missile  power  supply 

710  - MAS  - Missile  data  converter  (MDC) 

720  - MAS  - Missile  controller  (MC) 

601  - POWER  - Semiregulated  power  supply  (PWR) 
610  - POWER  - Regulated  power  supply  (REG/PWR) 
560  - C&D  - Hand  control  (HC) 

541  - DDD  - Detail  data  display 

580  - TID  - Tactical  information  display 


902  RACK: 


011  - XMTR  - Radar  transmitter 

006  - XMTR  - Transmitter  oscillator  wave  guide 

007  - CWI  - Continuous  wave  illuminator 

012  - RADAR  - Radar  output  wave  guide  (WG) 

021  - RADAR  - Radar  input  wave  guide  (WG) 

031  - ANT  - Radar  antenna 


025  - HORN  - Radar  test  horn 


101  - IR  - Infrared  receiver 

022  - RCVR  - Radar  receiver 

001  - RMO  - Radar  master  oscillator 

010  - RADAR  - Radar  synchronizer  (RAD/SYNCH) 


990  RACK: 


590  - MREC  - Mission  recorder 


962  RACK: 


462  - PWR/TAPE  - Power  supply/bulk  storer 
461  - CPIO  - Computer 

451  - AC/DRO  - Arithmetic  and  control/DRO 

452  - NDRO  - NDRO  memory 


I4O 
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LEFT  SIDE: 


015  - PWR  - Solenoid  power  supply,  radar  (SOL/PWR) 
120  - IR  - Infrared  amplifier 

013  - PWR  - Collector  power  supply  (COL/PWR) 

014  - PWR  - Beam  power  supply  (BEAM/PWR) 


560  - C&D  - Hand  control 

505  - C&D  - Computer  address  (ADDRESS) 

501  - C&D  - Sensor  control  (SENS/CON) 


081  - ANT/TC  - Antenna  and  test  controller 
083  - RADAR  - Low  PRF  processor  (LCW  PRF/PROC) 
039  - RADAR  - Doppler  processor  (DOPPL/PROC) 
042A  - DF  - Doppler  filter  (DOPPL/FIL) 

042b  - DF  - Doppler  filter  (DOPPL/FIL) 
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These  rear  cockpit  controls  and  displays  also  are  used  for  SPAM  and 
FLYCATCHER.  Manual  operations  and  information  displaye  are  i eren 
these  BEYOND  BIT  procedures. 

The  general  organization  of  the  BIT  sequences  is  illustrated  in  figure 
13  The  cLtinuous  monitoring  mode  is  used  by  the  NFO  (non-flying  officer) 
during  tactical  procedures.  Generally  speaking  computer 
the  AWG-9  system  for  running  BIT  and  run  each  BIT  ’ 

system  must  be  in  the  BIT  state  when  SPAM  tests  are  run.  SPAM.  Special 
Test  90,  generally  would  be  used  following  BIT,  to  perform  more  detailed 

analyses . 

The  first  four  BIT  sequences  are  confidence  tests.  If  they  reveal  no 
abnormal  states,  the  remaining  sequences  are  not  run.  BIT 
throuRht  eight  are  detailed  radar-IR  tests  that  are  run  if  BIT  3 or  BIT 
resulL  indicate  a malfunction  is  present  in  some  possible  fault 
the  radar  subsystems.  The  MAS/MOAT  confidence  test  (BIT  sequence  i® 
lot  included  in  current  planning  for  the  trainer,  since  this  is  concerned 
with  highly  specialized  missile  systems. 

The  primary  BIT  control  WRA’s  are  461.  the  central  processor  and  081, 
the  test  controller  logic  housed  with  the  antenna 

bit  sequence  programs  are  stored  on  magnetic  tapes  in  the  bulk-storer 
housed^in  WRA  462.  Some  BIT  sequences  require  intermediate  core  overlay  . 

There  are.  generally  speaking,  both  analogue 
between  the  CPU  and  the  functional  units  under  test  (FUT 
is  a digital  part  of  this  interface.  SSI  standing  for  Standard  Serial 
llterulel  Digital  -omputer  words  ar.  multiplexed 

over  this  loop.  Some  outputs  from  the  compcier  are  called  SOP  s (serial 
a BIT  program  command  results  in  an  analogue  output 

an  AO  (analogue  output).  The  returning  analogue  signal  is  called  an  I 
(analogue  input). 
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AMCS  = Airborne  Missile  Control  System 
MAS  = Missile  Auxiliaries  Subsystem 
MOAT  = Missile  On  Aircraft  Test 
FD  - Fault  Detection 
FI  = Fault  Isolation 

DMA  = Degraded  Mode  Assessment  (Basic) 
EDMA  = Expanded  Degraded  Mode  Assessment 

Figure  A-13.  BIT  Organization 


I DETAILED 
RADAR/ 
>1R  TESTS 
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Each  BIT  sequence  is  composed  of  a series  of  tests.  Each  test  is 
composed  of  manual  test  set-up  (or  initial ization^  which  results  in 
automatic  injection  of  some  type  of  signal  (or  using  noise  for  a signal) 
at  one  point  for  the  FUT,  measurement  of  the  output  at  another  point  for^^ 
the  FUT,  comparison  with  a reference,  and  making  the  decision,  "abnormal' 
or  "normal."  The  points  in  the  BIT  sequence  programs  at  which  this  deci- 
sion is  made  are  called  decision  points  (DP's).  These  DP's  are  numbered. 

An  abnormal  test  result  also  results  in  a DP  number  or  numbers  being  stored 
in  a buffer.  At  the  end  of  running  a BIT  sequence,  these  DP  numbers  are 
collected  and  interpreted  by  the  BIT  program  to  generate  the  symptom  infor- 
mation that  is  displayed  for  the  technician  on  the  TID  in  the  rear  cockpit. 


It  must  be  recognized  that  this  is  a highly  simplified  account.  For 
example  a particular  test  in  a BIT  sequence  may  be  run  several  times  be- 
fore a decision  point  is  reached,  and  an  entire  BIT  sequence  may  also  be 
run  several  times  in  succession  as  a part  of  the  maintenance  procedures. 
Also,  the  exact  kind  of  information  that  is  displayed  on  the  TID  difters 
in  important  details  from  one  BIT  sequence  to  the  next.  One  display  that 
is  common  to  BIT  sequences  2-4  is  the  Degraded  Mode  Assessment  (DMA).  This 
assessment  is  cumulative  from  sequence  2 on,  which  is  one  reason  why  BIT 

ml r\WA  i » 


in  figure 

14. 

The  acronyms  for  the  modes  have  the  following 

1. 

PDS  - 

Pulse  Doppler  Search 

2. 

RWS  - 

Range  While  Search 

3. 

TWS  - 

Track  While  Scan 

4. 

PDT  - 

Pulse  Doppler  Single  Target  Track 

5. 

BIT  - 

Built  In  Test 

6, 

PS  - 

Pulse  Search 

7. 

AG  - 

Air-to-Ground  (PS  with  Antenna  down) 

8. 

PT  - 

Pulse  Single  Target  Track 

9. 

ACM  - 

Air  Combat  Maneuver 

10. 

IRS  - 

IR  Scan  (displayed  if  IR  is  on  board) 

11. 

TVS  - 

TV  Scan  (displayed  if  TV  is  on  board)  . 

Special  tests,  e.g.,  SPAM  and  FLYCATCHER,  probably  are  more  subject 
to  further  development  by  the  contractor  as  the  weapon  system  matures. 
Consideration  of  these  for  the  trainer  will  be  deferred  until  later,  with 
the  exception  of  the  SPAM  memory  inspection  function.  Some  brief  notes 
on  SPAM  follow! 


SPAM  is  a contraction  of  SIPS,  SOPS,  PIPS,  POPS,  AOS,  AIS,  and  M^ORY. 
It  is  identified  as  Special  Test  90.  SPAM  is  one  type  of  test  that  would 
be  used  after  running  BIT.  Where  and  when  to  use  it  is  prescribed  in  the 
fault  localization  logic  trees  discussed  in  a previous  section. 

SPAM  contains  five  subroutines  that  allow  the  operator  to  examine  selec. 
ted  computer  memory  addresses,  inputs,  and  outputs  to  aid  in  maintenance 
(only  the  memory  inspection  function  would  be  implemented  in  the  trainer 

in  the  near  future); 
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1.  NDRO  (ron-destructive  readout)  memory  stability  test  (run  first: 
if  NDRO  is  faulty,  replace  before  proceeding  with  SPAM)  (figure  15)- 

2.  Serial  input/serial  output  (SIP/SOP)  subroutine. 

3.  Program  input/program  output  (PIP/POP)  subroutine:  controls  IFU 

(Interface  Unit)  functions. 

4.  Analogue  input/analogue  output  (AI/AO)  subroutine. 

5.  Memory  inspection  subroutine  (figure  16). 

It  is  emphasized  that  this  is  a very  brief  overview  of  the  AWG-9  WCS 
and  BIT,  intended  merely  to  acquaint  the  reader  with  some  general  features 
of  the  two  systems.  Full  documentation  for  these  systems  occupies  a 
number  of  large  technical  manuals.  The  contextual  structure  analysis  for 
the  development  of  data  structures  to  describe  essential  features  of  the 
BIT  sequences  and  tests  requires  an  electronics  engineer-senior  programmer 
team  to  analyze  each  test  within  a BIT  sequence  in  detail.  The  products 
of  these  analyses  are  data  structures  comprised  of  feature  lists  and  block 
diagrams  for  BIT  sequence  tests. 
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= Degraded  BCArD 

= Fail 

= Not  Evaluated 


Figure  A- 14.  Typical  BIT  Fault  Detection  Display 
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THE  5-DIGIT  (OCTAL) 
NUMBER  DISPLAYED  HERE 
REPRESENTS  A MEMORY 
LOCATION  IN  THE  NDRO 
(20000  THRU  77777). 


Figure  A-15.  NDRO  Memory  Stability  Test  Display 


Figure  A-I6.  Typical  Special  Test  90  Memory  Inspection  Routine  Display 


